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(54) White balance control apparatus 

(57) An imaging device (1R,1G,1B) is provided for 
converting an image into a plurality of color signals (R, 
G,B) each having a signal level, a white balance ampli- 
fier (5) adjusts the signal level of at least one color signal 
to produce a plurality of amplified color signals, and a 
calibration device (12) calibrates the white balance am- 
plifier (5) and produces at least one calibration param- 
eter. A detecting device detects the amplified color sig- 
nals, and a calculation device calculates at least one 



white balance amplification adjustment as a function of 
the plurality of amplified color signals. A comparing de- 
vice compares at least one white balance amplification 
adjustment with at least one calibration parameter, and 
an automatic adjustment device (12) adjusts the while 
balance amplifier (5) to amplify the signal level of at least 
one of the plurality of color signals if at least one white 
balance amplification adjustment is consistent with at 
least one calibration parameter. 



r-IR 2R 



CCD 

(R) 



S/H GAIN 
AOJUSTi\AENT 



'1G 2G 



,~3G 



S/H GAIN 
ADJUSTMENT 



IB 2B 



3B 



,~-4B 



CCD 
(B) 



S/H GAIN 
ADJUSTMENT 



-5R 



-5G 



5B 



100 



16 



15 — 



D/A 



OPERATION 
PART 



-5 



6R 



6G 



68 



ENCODER 



OOP' 

CD 



B 



OUTPUT 



A/D 



FT 



— 17 



CONTROLLER 
(MICROCOMPUTER) 



FBG.i 



14 



MEMORY 



CG 



COiViPOSITE 



10 



VF DRIVE 



11— VF 



13 



Printed by Jouve. 75001 PARIS (FR) 



BNSOOCID: <EP ^080031 8A2.L> 



1 EP 0 800 318 A2 



Description 

This invention relates to imaging apparatus, such 
as video cameras. 

In a color video camera, white balance is achieved 
when a neutral white object imaged by the camera under 
given illumination is represented as red (R), green (G). 
and blue (B) signals having equal output levels. White 
balance is needed since the RGB representation pro- 
duced by a color video camera typically changes as the 
Illumination of a scene varies. In some circumstances, 
a color video camera white balanced for certain illumi- 
nation conditions will not be white balanced for other il- 
lumination conditions. As a result, it is possible that an 
object under two different illuminations will have two dif- 
ferent RGB representations even though a human ob- 
sejver would perceive the object as having the same 
color under both illuminations. 

In a manual operation, white balance Is achieved 
by imaging a neutral white object under the illumination 
of interest and adjusting the amplification of one or more 
of the red, green, and blue signals until their respective 
output levels are equal. In an automatic white balance 
(AWB) operation, a neutral white object under the illu- 
mination of interest is imaged and the amplification lev- 
els of each of the red and blue signals are adjusted. For 
example, the output levels of the red and blue signals 
may be made equal to that of the green signal. In both 
operations, maintenance of white balance will depend 
upon the consistency of the illumination conditions and 
maintenance of the adjusted an-pl if ication levels. 

In an automatic tracing white balance (ATW) oper- 
ation, the white balance operation is automatically re- 
peatedly carried out during an ordinary imaging proc- 
ess. Since an ordinary imaged scene may not contain 
a neutral white object, it is possible that the white bal- 
ance will be incorrectly adjusted with reference to a 
colored object. Consequently, true white balance may 
not be achieved, e.g. a non-white color is represented 
as the color white. 

Figure 9 illustrates a black body radiation curve 
BBR. plotted on red signal gain vs. blue signal gain axes 
relative to red and blue signal amplifiers in a video cam- 
era. As shown, signal gain values are represented by 
eight-bits and, accordingly, each axis extends from 0 to 
256, The red signal gain and blue signal gain axes in- 
tersect at the point (1 28, 1 28). Also illustrated is a white 
area, indicated by slanted lining, which is generally sym- 
metric about the black body radiation cun^e BBR. The 
white area represents red and blue signal values which 
are characteristic of the color white. 

To avoid the problem of Incorrect white batance ad- 
justment during an ATW operation, the red and biue sig- 
nal values representing an imaged scene may be com- 
pared to a predetermined set of red and blue signal val- 
ues characteristic of the color white. The white area 
comprises sucn a set of red and blue signal values char- 
acteristic of the color white. 
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Previously, it was attempted to manually calibrate 
the white area such that output levels of the respective 
R. G. and B signals are mutually equal when a reference 
light source is imaged. With reference to Figure 9, cali- 

5 bration with respect to a standard light source would be 
attained when the B signal gain and the R signal gain 
are both equal to 128. 

In practice, a variable resistance was provided to 
manually adjust white balance amplification while a 

10 standard light source was imaged. A manual adjustment 
was made of the R signal and B signal output levels to 
make them equal to the G signal output level. Since the 
adjustment was made manually by a user or technician, 
errors in calibration have occurred and a precise adjust- 

15 ment has been difficult to achieve. Miscalibration intro- 
duces error into the ATW operation ^ reducing Its effec- 
tiveness. Additionally, the introduction of manual adjust- 
ment error may significantly limit further improvement in 
ATW processing efforts. Further error is introduced if the 

20 setting of the variable resistance shifts after a manual 
adjustment. Repeated manual readjustment can be dif- 
ficult. 

According to an aspect of the present invention, an 
imaging apparatus is provided which includes an imag- 
es ing device that converts an image into a plurality of color 
signals each having a signal level; a white balance am- 
plifier that adjusts the signal level of at least one of the 
color signals to produce a plurality of amplified color sig- 
nals; a calibration device that calibrates the white bal- 
30 ance amplifier and produces at least one calibration pa- 
rameter; a detecting device that detects the amplified 
color signals; a calculation device that calculates at least 
one white balance amplification adjustment as a func- 
tion of the amplified color signals; a comparing device 
35 that compares at least one white balance amplification 
adjustment with at least one calibration parameter; and 
an automatic adjustment device that automatically ad- 
justs the white balance amplifier to amplify the signal 
level of at least one color signal if at least one white bal- 
40 ance amplification adjustment Is consistent with at least 
one calibration parameter. 

According to another aspect of the present inven- 
tion, an imaging apparatus is provided which includes 
an imaging device for converting an image into a plural- 
45 ity of color signals each having a signal level; a white 
balance amplifier that adjusts the signal level of at least 
one color signal to produce a plurality of amplified color 
signals; a calibration device for repeatedly calibrating 
the white balance amplifier and for producing a plurality 
so of calibration parameters; a detectinq device that de- 
tects the amplified color signals; a first calculation de- 
vice that calculates at least one white balance amplifi- 
cation adjustment as a function of the ampliTiOd color 
signals; a storage device that stores the calibration pa- 
ss rameters; a second calculation device that retrieves the 
calibration parameters from the storage means and that 
calculates an overall calibration factor as a function of 
the plurality of calibration parameters; a comparing de- 
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vice that compares at least one white balance annplifi- 
catlon adjustment with the overall calibration parameter; 
and an automatic adjustment device for automatically 
adjusting the white balance amplifier to amplify the sig- 
nal level of at least one of the color signals if at least one 
white balance amplification adjustment Is consistent 
with the overall calibration parameter. 

A preferred embodiment of the invention provides 
apparatus and a method for an improved white balance 
operation, by means of an automatic white balance op- 
eration and an automatic tracing white balance opera- 
tion having improved accuracy and precision, and also 
an adaptive automatic tracing white balance operation 
for precisely adjusting white balance based upon the re- 
sults of previous automatic white balance operations. 

The invention will now be described by way of ex- 
ample with reference to the accompanying drawings, 
throughout which like parts are referred to by like refer- 
ences, and in which: 

Fig. 1 is a block diagram of an image recorder ac- 
cording to an embodiment of the present invention; 
Fig. 2 is a flow chart to which reference will be made 
in describing the operation of an image recorder 
embodying the present invention; 
Fig. 3 is a menu display diagram; 
Figs. 4(a) and 4(b) are diagrams of gain values; 
Figs. 5(a) and 5(b) are menu display diagrams; 
Fig. 6 is a diagram of gain values; 
Fig. 7 is a flow chart to which reference will be made 
in describing another operation of an image record- 
er embodying the present invention; 
Fig. 8 is a flow chart to which reference will be made 
in describing a step from the flow chart of Fig. 7; 
Fig. 9 is a diagram of gain values; 
Figs. 10(a) and 10(b) are diagrams of gain values; 
and 

Figs. 11 (a) and 11 (b) are diagrams of gain values. 

Figure 1 illustrates an imaging device 100 accord- 
ing to an embodiment of the present invention. Imaging 
device 1 00 includes charge-coupled devices (CCD) 1 R, 
1G, and IB; preamplifiers 2R, 2G. and 2B; sample/hold 
gain control circuits 3R, 3G, and 38; variable resistors 
4R and 48; white balance amplifier 5; amplifiers 6R, 6G, 
and 68; encoder 7; composite circuit 9, viewfinder driver 
10. viewfinder 11 , controller 1 2, character generator 1 3. 
memory device 1 4, digital-to-analog (D/A) converter 1 5, 
operation circuit 16; and analog-to-digital (A/D) convert- 
er 17. 

CCD IR, CCD 1G, and CCD IB are conventional 
primary color image pick-up devices for imaging incident 
red, green, and blue light, respectively. CCD 1 R produc- 
es a red signal R, CCD 1 G produces a green signal G, 
and CCD 1 B produces a blue signal B which are trans- 
mitted to recpective pre-amplifiers 2R, 2G. and 28. Pre- 
amplifiers 2R, 2G, and 2Bare conventional pre-amplifier 
devices for amplifying input signals. 



Pre-amplifiers 2R, 2G. and 2B output amplified R. 
G. and 8 signals to respective sample/hold gain control 
circuits 3R, 3G, and 38. Sample/hold gain control cir- 
cuits 3R, 3G, and 38 sample and hold input signals in 
s a conventional manner and pre-process the input image 
signals. For example, such preprocessing may include 
gain control, flare processing, pre-knee compensation, 
or the like. 

Pre-processed R and B signals from sample/hold 

10 gain control circuits 3R and SB are supplied to respec- 
tive variable resistors 4R and 48. Variable resistors 4R 
and 48 are variable resistance elements for further ad- 
justing the pre-processed Rand B signals, respectively. 
Variabte resistors 4R and 48 are utilized to adjust the 

75 levels of pre-processed R and B signals to achieve a 
white balance operatior\. 

White balance ampJIi^er 5 is comprised of variable- 
gain amplifiers SRand 58, along with amplifier 5G. Each 
of variable-gain amplifiers 5R and 58 are conventional 

20 variable-gain amplifiers, while amplifier 5G is a conven- 
tional amplifier. Variable -gain amplifier 5R amplifies the 
signal supplied from variable resistor 4R to produce an 
amplified R signal which is supplied to amplifier 6R. Var- 
iable-gain amplifier 58 amplifies the signal supplied 

25 from variable resistor 48 to produce an amplified B sig- 
nal which is supplied to amplifier 6B. Amplifier 5G am- 
plifies the pre-processed G signal supplied from sample/ 
hold gain control circuit 3G to produce an amplified G 
signal which is supplied to amplifier 6G. 

30 As a practical example, a white balance operation 
could be achieved by imaging a reference light source, 
setting the gains of variable-gain amplifiers 5R and 58 
at intermediate values, and then adjusting variable re- 
sistors 4R and 4B such that the outputs of amplifier 5 

35 are equal. However, in the present embodiment, prima- 
rily variable-gain amplifiers 5R and 58 are utilized to 
achieve the white balance operation. 

Amplifiers 6R, 6G, and 68 are conventional ampli- 
fier devices which further amplify the amplified R, G, and 

40 8 signals input respectively thereto, and which output 
further amplified R, G. and 8 signals to encoder 7. 

The amplified R, G, and B signals are also supplied 
to A/D converter 17. /VD converter 17 is a conventional 
analog-to-digital conversion device. A/D converter 17 

45 converts the amplified R, G, and 8 signals to digital R, 
G, and 8 signals, respectively which are supplied to 
controller 12. 

Controller 12 is a conventional controller device, 
preferably a conventional microprocessor device, which 

so operates to control imaging device 1 00. Controller 1 2 is, 
in turn, responsive to control signals provided by oper- 
ation circuit 16. alternatively, operation circuit 16 stores 
software programs which are accessed by and imple- 
mented within controller 12 to direct the operation of 

55 controller 1 2. Operation circuit 1 6 is preferably a source 
of control signals. Alternatively, operation circuit 16 is a 
memory device which stores software programs for di- 
recting the operation of controller 12. Controller 12 uti- 



5 



EP 0 800 318 A2 



6 



lizes memory 1 4, a conventional data storage device, in 
performing its control operations. Memory 14 may store 
parameter data, such as gain control values, for an AWB 
operation by controller 12. provide temporary data stor- 
age for controller 12 during computations and other 
processing, and perform additional similar activities. 

Encoder 7 is a conventional signal processing and 
encoding device. Encoder 7 may process the further 
amplified R. G, and 8 signals output by amplifiers 6R. 
6G, and 6B to achieve gamma correction, knee correc- 
tion, or the like. The input signals are then ma^ ix proc- 
essed to produce a brightness signal Y and color differ- 
ence signals R-Y and B-Y. Further, the brightness signal 
and the color difference signals are encoded by an NT- 
SC encoding process to produce a color composite vid- 
eo signal in accordance with the NTSC standard. The 
color composite video signal is output at output terminal 
8. The brightness signal and the color difference signals, 
or alternatively, corresponding image signals of another 
form, are also supplied to composite circuit 9. 

Character generator 1 3 is a conventional character 
generation device for generating character image sig- 
nals in response to character commands from controller 
12. Character generator 13 supplies character Image 
signals to composite circuit 9. Composite circuit 9 is a 
composition device for superimposing character image 
signals from character generator 1 3 onto image signals 
from encoder 7 to produce a composite signal. 

The composite signal is supplied to view finder driv- 
er 10, a conventional view finder driving device. View 
finder driver 1 0 drives view finder 1 1 to display an Image 
corresponding to the composite signal to a user. View 
finder 11 is a conventional view finder-type display de- 
vice. 

Controller 12 detects the respective signal levels of 
the amplified R, G, and B signals and controls imaging 
device 100 to achieve both AWB and ATW operations 
to produce an improved white balance control. In oper- 
ation, as a function of the respective signal levels of the 
amplified R, G, and B signals, controller 12 adjusts the 
respective gains of variable-gain amplifiers 5R and 5B. 
Specifically, controller 12 supplies digital gain control 
signals to D/A converter 15, a conventional digital-to- 
conversion device, which converts the digital signals to 
analog form. The analog signals are supplied to varia- 
ble-gain amplifiers 5R and 5B to control their amplifica- 
tion. Thus, A/D converter 1 7, controller 1 2. and D/A con- 
verter 15 comprise a feedback control system for con- 
trolling amplifier 5. 

To display information, such as operational state in- 
formation, display mode, and the like, to a user, control- 
ler 1 2 supplies character commands corresponding to 
such information to character generator 13. Character 
generator 13 generates character image signals corre- 
sponding to the supplied character commands and out- 
puts such signals to composite circuit 9. Composite cir- 
cuit 9 superimposes the character image signals on im- 
age signals supplied from encoder 7 to produce a com- 



posite signal for output to view finder driver 10. View 
finder driver 1 0 drives view finder 1 1 to display the com- 
posite signal to the user. Accordingly the user may mon- 
itor the imaged scene and the effect of the white balance 

5 processing, and view informative text. 

In an AWB operation, c itroller 1 2 monitors the sig- 
nal levels of the amplified v-^. G. and B signals while a 
white reference object Is imaged. Controller 1 2 then cal- 
culates the amplification amount (gain) needed to equal- 

10 ize each of the amplified R and B signals with the am- 
plified G signal. Corresponding gain control signals are 
supplied by controller 12, through D/A converter 15 to 
variable-gain amplifiers 5R and SB, respectively. In this 
manner, each of variable-gain amplifiers 5R and 5B are 

IS independently adjusted by controller 12 and white bal- 
ance is achieved at the outputs of amplifier 5. 

In an ATW operation, controller 12 consults a pre- 
determined set of red and blue signal values character- 
istic of the color white. Figures 4(a) and 4(b) illustrate 

20 an ATW control area which comprises such a set of red 
and blue signal values characteristic of the color white, 
along with a black body radiation curve BBR. As shown, 
the ATW control area, indicated by slanted lining, is pref- 
erably symmetric about black body, radiation curve 

25 BBR. 

The ATW control area and the cun/e BBR are plot- 
ted on red signal gain vs. blue signal gain axes relative 
to the respective gains of variable-gain amplifiers 5R 
and 5B. Signal gain values are represented by eight-bits 

30 and, accordingly, each axis extends from 0 to 256, The 
red signal gain and blue signal gain axes intersect at the 
point (128, 128). Preferably, the ATW control area is 
centered at the point (128,128), as shown in Figure 4 
(a), which would correspond to a color temperature of 

35 3200 K. 

To initially calibrate the ATW control area an AWB 
operation is executed, preferably during the manufac- 
turing process. A standard light source is imaged by 
CCD's 1 R, 1 G, and 1 B. The corresponding R, G, and B 

40 signals are transmitted to pre-amplifiers 2R. 2G, and 2B, 
respectively, which amplify the respective signals. Sam- 
ple/hold gain control circuits 3R, 3G, and 3B sample and 
hold the pre-amplified R, G. and B signals, respectively, 
and pre-process the resulting signals to produce pre- 

45 processed R, G, and B signals, respectively 

The pre-processed R and B signals are supplied to 
variable resistors 4R and 4B, respectively. Preferably 
variable resistors 4R and 4B are initially set in a fixed, 
e.g. constant resistance, state. Variable resistors 4R 

50 and 4B adjust the respective levels of the pre-processed 
R and B signals and supply the adjusted R and B signals 
to variable-gain amplifiers 5R and 5B, respectively The 
pre-processed G signal is supplied to variable-gain am- 
plifier 5G. 

55 Variable-gain amplifiers 5R, 5G, and 5B provide an 
initial amount of amplification to each of the input R. G, 
and B signals to produce amplified R, G. and B signals, 
respectively. Controller 12, through A/D converter 17, 
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detects the output levels of the amplified R, G, and B 
signals and calculates the gain adjustments needed by 
variable-amplifiers 5R and 5B, respectively, to equalize 
the amplified R and B signals with the amplified G signal. 
Controller 1 2 stores the gain adjustment values in mem- 
ory 14 and supplies corresponding gain control signals 
to variable-amplifiers 5R and 58 through D/A converter 
15. The respective amplification levels of variable-am- 
plifiers 5R and 5B are adjusted accordingly 

It is noted that the above-described AWB operation 
implements a feedback loop which omits any adjust- 
ment of variable resistors 4R and 4B, Preferably, varia- 
ble resistors are retained for other adjustment opera- 
tions. Optionally, the variable resistors may be omitted 
entirely. Alternatively, variable resistors 4R and. 4B may 
be utilized to coarsely adjust the signal levels of the R 
and B signals input to amplifier 5, after which the feed- 
back loop is utilized to achieve precise white balance 
calibration. 

Following calibration of the ATW control area, an 
ATW operation can be executed. As will be described 
below, the gain adjustment values calculated in a pre- 
ceding AWB operation are utilized during the ATW op- 
eration to calibrate the ATW control area prior to ATW 
processing. The AWB operation may be an initial AWB 
operation executed during the manufacturing of Imaging 
device 100 or may be a subsequent AWB operation ex- 
ecuted by a user. 

In an ATW operation, preferably, controller 12 is- 
sues to character generator 1 3 character commands 
corresponding to a menu of user functions. Character 
generator 13 generates character image signals corre- 
sponding to the menu and outputs such signals to com- 
posite circuit 9. Composite circuit 9 superimposes the 
menu on image signals supplied from encoder 7 to pro- 
duce a composite signal for output to view finder driver 
1 0. View finder driver 1 0 drives view finder 1 1 to display 
the composite signal which includes the menu to the us- 
er. 

Figure 3 illustrates a preferred ATW operation 
menu. As shown, the user is displayed a prompt to ini- 
tiate an ATW adjustment operation ("ATW ADJ"). When 
the ATW adjustment operation is selected by the user, 
the R and B signal gain adjustments calculated by con- 
troller 12 In a preceding AWB operation are then dis- 
played and stored by controller 1 2 in memory 1 4 as sig- 
nal gain adjustments Radj and Badj, respectively. Op- 
tionally, the R and B signal gain adjustments are initially 
displayed along with the ATW adjustment operation 
prompt. In this manner, subsequent AWB operations 
may be utilized to update stored Radj and Badj values. 

Preferably, the user's selection of the ATW adjust- 
ment operation is entered via a user interface portion of 
operation circuit 16. Operation circuit 16 transmits the 
user's selection to controller 1 2 which initiates the ATW 
adjustment operation accordingly. The ATW adjustment 
operation may be carried out repeatedly by a user as 
needed. 



Figure 2 illustrates an ATW adjustment operation 
method according to an embodiment of the present in- 
vention. In step F101, controller 12 detects the output 
signal levels of the amplified R, G. and B signals via N 

5 D converter 17, and processing proceeds with step 
F102. In step F 102, controller 12 calculates the gain ad- 
justments, Rga!n Bqain. needed by variable-ampli- 
fiers 5R and SB. respectively, to equalize the amplified 
R and B signals with the amplified G signal, and 

10 processing proceeds with step F103. 

In step F103. an offset for the ATW control area is 
calculated by subtracting the ideal intermediate amplifi- 
cation value, e.g. 1 28 in the present example, from each 
of the stored signal gain adjustments Radj and Badj. As 

IS an example, utilizing sample Radj and Badj values from 
Figure 3, Radj = 131 and Badj = 128, an R gain offset 
of 3 (=1 31 -1 28) and a B gain offset of 0 (=1 28-1 28) can 
be calculated. Controller 12 corrects (shifts) the ATW 
control area by an amount equal to the calculated R gain 

20 offset and B gain offset values. Further to the above ex- 
ample, correction of the ATW control area of Figure 4(a) 
is achieved by shifting the ATW control area by three R 
gain units and zero B gain units as shown in Figure 4 
(b). Of course, the ATW control area may be shifted 

25 along the Rgain axis, the Bgain axis, or both depending 
on the calculated offset values. Processing proceeds 
with step F 104. 

In step F104, the gain adjustments, Rqain 
^GAiN' calculated by controller 1 2 are compared by con- 

30 troller 12 to the corrected ATW control area. If either 
f^GAiN Bqain '® outside the corrected ATW control ar- 
ea, i.e. represent values not included among the set of 
values represented by the ATW control area, then con- 
troller 1 2 determines that the present corresponding im- 

35 age is not suitable for use in white balance adjustment, 
amplifiers 5R and 5B are not adjusted, and processing 
returns to step F101 . If both Rqain ^gain are within 
the corrected ATW control area. i.e. represent values 
included among the set of values represented by the 

40 ATW control area, then controller 1 2 determines that the 
present corresponding image is suitable for use in while 
balance adjustment and processing proceeds with step 
F105. 

In step F105, controller 1 2 supplies gain control sig- 
45 nals corresponding to the Rqain ^gain values 
through D/A converter 1 5 to variable-gain amplifiers 5R 
and 5B, respectively. In this manner white balance is 
achieved. The white balanced signals are amplified by 
amplifiers 6R. 6G, and 6B, encoded by encoder 7, and 
50 output at output terminal 8. 

In an alternate ATW operation, variation in the ref- 
erence light source used for white balance calibration 
can be accommodated by manual adjustment of the 
ATW control area. VSariation in the reference light source 
55 may be measured by a user with a color chromaticity 
system and the resulting measurements converted into 
corresponding signal gain adjustment values. Radj and 
Badj for correction (shift) of the ATW control area. 
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Figures 5(a) and 5(b) illustrate an alternative ATW 
operation menu. As in the menu of Figure 3, the user Is 
displayed a prompt to initiate an ATW adjustment oper- 
ation ("ATW ADJ") along with the R and B signal gain 
adjustments calculated by controller 12 in a preceding 
AWB operation. Additionally, the user is prompted to 
modify, if desired, the displayed R and B signal gain ad- 
justment values. The user enters a desired modification 
via the user interface portion of operation circuit 1 6. Fig- 
ure 5(a) shows a user's modification of the R signal gain 
adjustment value to 1 35 and Figure 5(b) shows a user's 
modification of the B signal gain adjustment value to 
130. The modified R and B signal gain adjustments are 
stored by controller 12 in memory 14 as signal gain ad- 
justments Radj and Badj, respectively, e.g. Radj = 135 
and Badj = 130. Of course, the preceding values are 
intended to illustrate and not limit the scope of the 
present invention. 

Otherwise, ATW processing proceeds as illustrated 
in Figure 2. Utilizing the sample modified signal gain ad- 
justment values mentioned above, the ATW control area 
of Figure 4(a) is correspondingly corrected by shifting 
the area by seven R gain units and two B gain units as 
shown in Figure 6. 

In a further embodiment of imaging device 1 00, con- 
troller 1 2 stores multiple sets of R signal gain adjustment 
and B signal gain adjustment values produced by suc- 
cessive implementation of the above-described AWB 
operation and/or manual adjustment of the ATW control 
area. For example, Radj and Badj values from a previ- 
ous ten AWB operations may be stored in memory 14 
for selective recall by a user. 

Preferably, sets of R signal gain adjustment and B 
signal gain adjustment values, Radj and Badj, are stored 
in memory 14 as (Rawb[1] , Bawb[1]), (Rawb[2], Bawb 
[2])...(Rawb[10], Bawb[10]) in reverse chronological or- 
der, where "(RawbM. BawbM)" (x = 1. 2,. .10) repre- 
sents a pair of registers, or simply address locations, in 
memory 14. Preferably, after each AWB operation or 
manual adjustment of the ATW control area, the oldest 
R signal gain adjustment value and the oldest B signal 
gain adjustment value, e.g. the values stored in (Rawb 
[10], Bawb[10]), are discarded; each of the remaining 
pairs of values are shifted by one register (address lo- 
cation is incremented); and the new R and B signal gain 
adjustment values are stored in (Rawb[1] , Bawb[11). In 
this manner, the reverse chronological order is pre- 
served. 

A method of ATW processing with storage of multi- 
ple signal gain adjustment values is illustrated in Figure 
7. In step F201, processing is executed as described 
above with respect to step F101 but is followed by step 
F202. In step F202, processing is executed as da- 
scribed above with respect to step F102 but is followed 
by step F203. 

In step F203, an offset for the ATW control area is 
calculated as a function of the distribution of the pairs 
of Radj and Badj values stored in memory 14, e.g. 



(Rawb[1], Bawb[1]). (Rawb[2], Bawb[2])...(Rawb(10]. 
Bawb[10]) . This calculation will be described in further 
detail hereinbelow. Controller 12 corrects (shifts) the 
ATW control area by an amount equal to the calculated 

5 R gain offset and B gain offset values. 

In subsequent step F204, if either Rqain or Bq^in »s 
outside the corrected ATW control area, i.e. represent 
values not included among the set of values represent- 
ed by the ATW control area, then controller 12 deter- 

10 mines that the present corresponding image is not suit- 
able for use in white balance adjustment, amplifiers 5R 
and 5B are not adjusted, and processing returns to step 
F201 . If both Rqain ^omh a*"© within the corrected 
ATW control area, i.e. represent values included among 

15 the set of values represented by the ATW control area, 
then controller 12 determines that the present corre- 
sponding image is suitable for use in white balance ad- 
justment and processing proceeds with step F205. 
In step F205, processing is executed as described 

20 above with respect to step F105 but is followed by step 
F201. 

The offset calculation of step F203 will be described 
in further detail in connection with the flow chart of Fig- 
ure 8 and the diagram of Figure 10(b). Figure 10(b) il- 

25 lustrates a portion of the ATW control area shown in Fig- 
ure 4(a) and shows, specifically, nine pairs of Radj and 
Badj values represented as points SI , S2, S3, S5, S6... 
S10. For the purposes of explanation, and not as a lim- 
itation on the present invention, it is assumed that mem- 

30 ory 1 4 has stored ten pairs of Radj and Badj values cor- 
responding to points SI , S2,...S10 as follows: 

SI : (127,127) is stored at (Rawbfl], Bawb[1]) 
S2: (129,126) is stored at (Rawb[2], Bawb[2]) 

35 S3: (144.108) is stored at (Rawb[3] , Bawb[3]) 
S4: (241,200) is stored at (Rawb[4], Bawb[4]) 
S5: (120,130) is stored at (Rawb[5], Bawb[5]) 
S6: (123,134) is stored at (Rawb[6], Bawb[6]) 
S7: (142,119) is stored at (Rawb[7], Bawb[71) 

40 S8: (1 08, 1 1 8) is stored at (Rawb[8], Bawb[8]) 
S9: (121 ,146) is stored at (Rawb[9]. Bawb[9]) 
S10: (127.129) is stored at (Rawb(10], Bawb[10]) 

Of course, another positive number of pairs of Radj and 
45 Badj values other than ten could be utilized equally ef- 
fectively,. 

It is preferable to define a sample area SA within 
the ATW control area as corresponding to a particular 
type of lighting. For example. Figure 10(b) illustrates a 

50 sample area SA which may correspond to light h ; ng 
an approximate temperature of 3200 K, e.g. Rao; id 
Badj have values greater than 120 but less thari - 6. 
Alternatively, sample area SA may have another -pe 
such as a circle, an ellipse, or the like. 

55 As shown, points SI , S2, S5, S6. and SI 0 lie ... hin 
sample area SA, while points S3, S4, S7, S8, and S9 lie 
outside sample area SA. The points inside sample area 
SA may be considered similar to the desired type of 
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lighting, and therefore, useful in adjusting the ATW con- 
trol area. The points outside sample area SA may be 
considered too different from the desired type of lighting 
to be useful in adjusting the ATW control area. By sep- 
arately averaging the Radj and Badj values represented 
by the points inside sample area SA. an average R gain 
adjustment value (Rav) and an average B gain adjust- 
ment value (Bav) can be calculated. These average val- 
ues can then be used as offsets to correct (shift) the 
ATW control area. 

Figure 8 illustrates steps F231 - F239 which form 
step F203 by which controller 12 determines which 
stored points fall within the desired range and separately 
averages the values corresponding to such points. In 
step F231 , a variable n is initialized by controller 1 2 with 
the value 1 . Next, in step F232 a variable m is initialized 
by controller 12 with the value 1. Processing proceeds 
with step F233. 

In step F233, controller 12 determines whether 
point Sn, i.e. (Rawb[n]. Bawb[n]), is located within the 
desired sample area SA. If Sn is within SA, processing 
proceeds with step F234; otherwise, processing pro- 
ceeds with step F236. 

In step F234, the value stored at Rawb[n] is tempo- 
rarily stored at another address, or register, Rsp[m], as 
a desired sampled R gain adjustment value. Similarly, 
the value stored at Bawb[n] is temporarily stored at an- 
other address, or register, Bsp[m], as a desired sampled 
B gain adjustment value. In the next step F235, the var- 
iable m is incremented by 1 and processing proceeds 
with step F236. 

In step F236. if the variable n is less than the 
number of points to be considered, e.g. 1 0. then variable 
n is incremented by 1 and processing proceeds with 
step F233. Otherwise, processing proceeds with step 
F240 in which m is decremented by 1 . Utilizing the data 
and sample area SA provided in Figure 10(b) as an ex- 
ample, the desired sampled R and B gain adjustment 
values determined by controller 12 would be stored In 
memory 1 4 as follows: 



Rsp[1] = 


127, Bsp[1] = 


1 27 (corresponding to point 


SI) 






Rsp[2] = 


129, Bsp[2] = 


1 26 (corresponding to point 


S2) 






Rsp[3] = 


120. Bsp[3] = 


1 30 (corresponding to point 


S5) 






Rsp[4] = 


123. Bsp[4] = 


1 34 (corresponding to point 


S6) 






Rsp[5] = 


127, Bsp[5] = 


1 29 (corresponding to point 


S10) 







Following step F240, in step F238. controller 21 cal- 
culates an average R gain adjustment value (Rav) as 
the average of the m desired sampled R gain adjustment 
values. For example, the desired sampled R gain ad- 
justment values are summed and the sum is divided by 
the number m to produce Rav. Similarly, controller 21 



12 

calculates an average B gain adjustment value (Bav) as 
the average of the m desired sampled B gain adjustment 
values. For example, the desired sampled B gain ad- 
justment values are summed and the sum is divided by 
s the number m to produce Bav. Utilizing the desired sam- 
ple gain adjustment values provided in the example 
above, 

Rav = (127 + 129 + 120 + 123 + 127)/5 = 125 

and 

Bav = (127 -I- 126 + 130 -i- 134 + 129)/5 = 129. 

Processing proceeds with step F239. 

In step F239. an offset for the ATW control area is 
20 calculated by subtracting the ideal intermediate amplifi- 
cation value, e.g. 1 28 in the present example, from each 
of the average gain adjustment values Rav and Bav. 
Continuing the example above, an R gain offset of -3 
(=125-1 28) and a B gain offset of 1 (=129-128) can be 
25 calculated. Controller 1 2 corrects (shifts) the ATW con- 
trol area by an amount equal to the calculated R gain 
offset and B gain offset values. Further to the above ex- 
ample, correction of the ATW control area of Figure 4(a) 
is achieved by shifting the ATW control area by three R 
.30 gain units (to the left) and one B gain unit as shown in 
Figure 10(a). 

By sampling past gain adjustment values, controller 
12 estimates the most acceptable correction amount 
(shifting) for, the ATW control area to enable proper 

3S white balance operation under what is expected to be 
simitar future lighting conditions. Controller 12 thus 
"learns" how much to adjust the ATW control area based 
upon past experience with previous AWB operations 
and/or manual ATW control area adjustments. Conse- 

40 quently, the white balance operation is optimized for a 
particular image based upon past white balance results 
and an improved white balance operation is achieved. 

It is further noted that statistically, in imaging a cer- 
tain image, periodic AWB operations will produce pairs 

4S of R and B signal gain adjustment values which have a 
distribution similar to that of a black body radiation 
cun/e. Thus, the historical values of the pairs of R and 
B signal gain adjustment values are an advantageous 
basis for an ATW control area. Upon repeated ATW op- 

50 erations, the ATW control area will be adjusted to center 
on the distribution of past pairs of R and B signal gain 
adjustment values. 

In an alternate embodiment, step F240 is modified 
such that controller 21 , after having decremented the 

55 variable m by one, determines if m is greater than a min- 
imum value mv. For example, mv may equal five. If m is 
greater than mv, processing is proceeds with step F238; 
othenwise, no adjustment of the ATW control area is 
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made at that time and processing proceeds with step 
F201 . Such a feature limits the frequency of ATW control 
area adjustment until a minimum number of images hav- 
ing a relatively consistent pattern of lighting are imaged. 
Overall ATW control area adjustment and thus, white 
balance operation, are thereby improved. 

As an alternative to the above-described process- 
ing of step F203, controller 12 may instead determine 
from the R signal gain adjustment and B signal gain ad- 
justment values, Radj and Badj, stored in memory 1 4 as 
(Rawb[1], Bawb[1]). (Rawb[2]. Bawb[2])...(Rawb[x]. 
Bawb[x]), the R gain offset and B gain offset values 
which will result in the corrected ATW control area en- 
compassing the most pairs of stored R and B signal gain 
adjustment values. For example, Figure 11(a) illustrates 
ten points representing pairs of stored R and B signal 
gain adjustment values and an ATW control area. A 
maximization of the number of points encompassed by 
the ATW control area is illustrated in Figure 11(b). This 
maximization may be achieved by a number of different 
conventional methods, such as a line-fit, a curve fit to 
black body radiation cun/e BBR, an area-fit to a sample 
number of points, or th ^ like. 

As will be understood by one of ordinary skill in the 
art, the term amplify, and synonyms thereof, utlilized in 
the present specification refer to and encompass both 
the process of amplification and de-ampliflcation. Also, 
color signals other than the red color signal and the blue 
color signal described herein, such as a green color sig- 
nal, a yellow color signal, or the like, may be adjusted 
by the invention of the present application without de- 
parting from its scope. 

Although illustrative embodiments of the present in- 
vention and modifications thereof have been described 
in detail herein, it is to be understood that this invention 
is not limited to these precise embodiments and modi- 
fications, and that other modifications and variations 
may be effected therein by one skilled in the art without 
departing from the scope of the invention as defined by 
the appended claims. 



Claims 

1 . An imaging apparatus comprising: 

imaging means for converting an image into a 
plurality of color signals each having a signal 
level; 

white balance amplifier means, coupled to said 
Imaging means, for adjusting the signal level of 
at least one of said plurality of color signals to 
produce a plurality of amplified color signals; 
calibration means for calibrating said white bal- 
ance amplifier means and for producing at least 
one calibration parameter; 
detecting means, coupled to said white balance 
amplifier means, for detecting said plurality of 



amplified color signals; 

calculation means, coupled to said detecting 
means, for calculating at least one white bal- 
ance amplification adjustment as a function of 
5 said plurality of ampUfied co^or signals; 

comparing means for comparing said at least 
one white balance amplification adjustment 
with said at least one calibration parameter; 
and 

10 automatic adjustment means for automatically 

adjusting said white balance amplifier means to 
amplify the signal level of at least one of said 
plurality of color signals if said at least one white 
balance ampfification adjustment is consistent 

15 with said at least one calibration parameter 

2. Apparatus according to claim 1 . further comprising 
storage means for storing said at least one calibra- 
tion parameter and wherein said comparing means 

20 retrieves said at least one calibration parameter 
from said storage means. 

3. Apparatus according to claim 1 , wherein said cali- 
bration means comprises means for automatic 

25 white balancing. 

4. Apparatus according to claim 1 , wherein said cali- 
bration means comprises at least one user-adjust- 
able signal adjustment means for adjusting the sig- 

30 nal level of at least one of said plurality of color sig- 
nals. 

5. Apparatus according to claim 1, wherein said at 
least one calibration parameter comprises a piural- 

35 ity of calibration parameters. 

6. Apparatus according to claim 1, wherein said at 
least one calibration parameter comprises a red 
color gain adjustment value. 

40 

7. Apparatus according to claim 1, wherein said at 
least one calibration parameter comprises a blue 
color gain adjustment value, 

45 8. Apparatus according to claim 1, wherein said at 
least one calibration parameter comprises a green 
color gain adjustment value. 

9. Apparatus according to claim 1. wherein said at 
50 least one calibration parameter is characteristic of 

a black body radiation pattern. 

10. Apparatus according to claim 1, wherein said at 
least one calibration parameter defines an ATW 

55 control area which is substantially symmetric about 
a black body radiation pattern. 

1 1 . Appparatus according to claim 1 0, wherein said au- 
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tomatic adjustment means automatically adjusts 
said white balance amplifier means to amplify the 
signal level of at least one of said plurality of color 
signals if said at least one white balance amplifica- 
tion adjustment is within said ATW control area. 

1 2. Appparatus according to claim 1 0. wherein said au- 
tomatic adjustment means automatically does not 
adjust said white balance amplifier means if said at 
least one white balance amplification adjustment is 
outside said ATW control area. 

13. An imaging apparatus comprising: 

imaging means for converting an image into a 
plurality of color signals each having a signal 

level; 

white balance amplifier means, coupled to said 
imaging means, for adjusting the signal level of 
at least one of said plurality of color signals to 
produce a plurality of amplified color signals; 
calibration means for repeatedly calibrating 
said white balance amplifier means and for pro- 
ducing a plurality of calibration parameters; 
detecting means coupled to said white balance 
amplifier means, for detecting said plurality of 
amplified color signals; 

first calculation means, coupled to said detect- 
ing means, for calculating at least one white 
balance amplification adjustment as a function 
of said plurality of amplified color signals; 
storage means for storing said plurality of cali- 
bration parameters; 

second calculation means for retrieving said 
plurality of calibration parameters from said 
storage means and for calculating an overall 
calibration factor as a function said plurality of 
calibration parameters; 

comparing means for comparing said at least 
one white balance amplification adjustment 
with said overall calibration parameter; and 
automatic adjustment means for automatically 
adjusting said white balance amplifier means to 
amplify the signal level of at least one of said 
plurality of color signals if said at least one white 
balance amplification adjustment is consistent 
with said overall calibration parameter. 

14. Apparatus as in claim 13, wherein said second cal- 
culation means calculates said overall calibration 
factor as an average of said plurality of calibration 
parameters. 

15. Apparatus as in claim 1 3, wherein said second cal- 
culation means calculates said overall calibration 
factor as an average of calibration parameters cor- 
responding to a color. 



16. Apparatus as in claim 1 3, wherein said second cal- 
culation means calculates said overall calibration 
factor as a function of a distribution of said plurality 
of calibration parameters. 

5 

17. Apparatus as in claim 1 3, wherein said second cal- 
culation means calculates said overall calibration 
factor as a function of a plurality of calibration fac- 
tors which are representative of a lighting condition. 

10 

18. Apparatus according to claim 13, wherein said cal- 
ibration means comprises means for automatic 
white balancing. 

IS 19. Apparatus according to claim 1 3, wherein said cal- 
ibration means comprises at least one user-adjust- 
able signal adjustment means for adjusting the sig- 
nal level of at least one of said plurality of color sig- 
nals. 

20 

20, Apparatus according to claim 1 3, wherein said plu- 
rality of calibration parameters comprises a red 
color gain adjustment value and a blue color gain 
adjustment value. 

25 

21. Apparatus according to claim 13, wherein said.plu- 
rality of calibration parameters are characteristic of 
a black body radiation pattern. 

30 22. Apparatus according to claim 1 3, wherein said plu- 
rality of calibration parameters define an ATW con- 
trol area which is substantially symmetric about a 
black body radiation pattern. 

35 23. Appparatus according to claim 22, wherein said au- 
tomatic adjustment means automatically adjusts 
said white balance amplifier means to amplify the 
signal level of at least one of said plurality of color 
signals if said at least one white balance amplifica- 

40 tion adjustment is within said ATW control area, 

24. Appparatus according to claim 22, wherein said au- 
tomatic adjustment means automatically does not 
adjust said white balance amplifier means if said at 
45 least one white balance amplification adjustment is 
outside said ATW control area. 
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(54) White balance control apparatus 



(57) An imaging device (1 R.I G. 1 B) is provided for 
converting an image into a plurality of color signals (R, 
G.B) each having a signal level, a white balance ampli- 
fier (5) adjusts the signal level of at least one color signal 
to produce a plurality of amplified color signals, and a 
calibration device (12) calibrates the white balance am- 
plifier (5) and produces at least one calibration param- 
eter. A detecting device detects the amplified color sig- 
nals, and a calculation device calculates at least one 



white balance amplification adjustment as a function of 
the plurality of amplified color signals. A comparing de- 
vice compares at least one white balance amplification 
adjustment with at least one calibration parameter, and 
an automatic adjustment device (12) adjusts the while 
balance amplifier (5) to amplify the signal level of at least 
one of the plurality of color signals if at least one white 
balance amplification adjustment is consistent with at 
least one calibration parameter. 
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